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Comparing networks in neuroscience is hard, because the topological properties of a given 
network are necessarily dependent on the number of edges in that network. This problem 
arises in the analysis of both weighted and unweighted networks. The term density is 
often used in this context, in order to refer to the mean edge weight of a weighted 
network, or to the number of edges in an unweighted one. Comparing families of networks 
is therefore statistically difficult because differences in topology are necessarily associated 
with differences in density. In this review paper, we consider this problem from two 
different perspectives, which include (i) the construction of summary networks, such as 
how to compute and visualize the summary network from a sample of network-valued 
data points; and (ii) how to test for topological differences, when two families of networks 
also exhibit significant differences in density. In the first instance, we show that the 
issue of summarizing a family of networks can be conducted by either adopting a 
mass-univariate approach, which produces a statistical parametric network (SPN). In the 
second part of this review, we then highlight the inherent problems associated with 
the comparison of topological functions of families of networks that differ in density. In 
particular, we show that a wide range of topological summaries, such as global efficiency 
and network modularity are highly sensitive to differences in density. Moreover, these 
problems are not restricted to unweighted metrics, as we demonstrate that the same 
issues remain present when considering the weighted versions of these metrics. We 
conclude by encouraging caution, when reporting such statistical comparisons, and by 
emphasizing the importance of constructing summary networks. 



Keywords: networks, N-back, statistical parametric network (SPN), small-world topology, working memory, 
weighted density, density-integrated metrics 



1. INTRODUCTION 

Are neurological networks topologically stable across different 
populations of subjects or across different cognitive and behav- 
ioral tasks? This general research program has been carried out 
by a myriad of researchers in the last decade. Neuroscientists 
are often interested in evaluating whether the small- wo rid prop- 
erties of a given brain network are conserved when comparing 
patients with controls. Bassett et al. (2008), for instance, have 
studied the differences in anatomical brain networks exhibited 
by healthy individuals and patients with schizophrenia. Similarly, 
some authors have tested how the topological properties of cer- 
tain functional networks are affected by different behavioral tasks 
(Cecchi et al, 2007; De Vico Fallani et al, 2008; van den Heuvel 
et al., 2009). Brain network topology has been studied at differ- 
ent spatial scale (Bassett et al., 2006), and different time scales 
(Pachou et al, 2008; Salvador et al, 2008). It is therefore unde- 
niable that there is considerable academic interest in comparing 
families of networks; whether these represent several groups of 
subjects, or the different conditions of an experiment. This gen- 
eral research paradigm is particular amenable to the analysis of 
subject- specific networks. When such individual networks are 



available, one can readily compute subject- specific topological 
measures, which will then be compared across experimental con- 
ditions. This type of analysis has been conducted using both 
functional and structural MRI data (Hagmann et al., 2008; Gong 
et al, 2009). In this paper, we will mostly focus on networks 
arising from functional MRI (fMRI) data. 

The prospect of performing rigorous statistical analysis of sev- 
eral populations of networks, however, has been hindered by 
various methodological issues. These statistical questions have 
not been hitherto satisfactorily resolved in the neuroscience com- 
munity, and the field of network data analysis remains an area 
of active methodological development (Simpson et al, 2013a,b). 
When one is considering the question of comparing several pop- 
ulations of brain networks, two main problems arise. First and 
foremost, the problem of the inherent dependence between con- 
nectivity strength (i.e., wiring density) and network topology (i.e., 
patterns of edges) necessarily arises. Most, if not all, of the topo- 
logical metrics that have become popular in the neuroscience 
literature are highly sensitive to the differences in the number of 
edges of the graphs under comparison. Therefore, when trying to 
evaluate the topological properties of different groups of networks 
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on the sole basis of their topology, one also requires to apply some 
level of control on the differences in density between the groups 
of networks under scrutiny. 

Secondly, the issue of separating differences in density from 
differences in topology is compounded by the problem of thresh- 
olding association matrices. In many cases, neuroscientists are 
considering correlation matrices with values ranging between — 1 
and 1 . Because network science is founded on graph theory, which 
is a branch of discrete mathematics, it follows that the applica- 
tion of graph-theoretical methods requires the use of a particular 
threshold in order to produce adjacency matrices. Naturally, this 
choice of threshold is often arbitrary, although various statisti- 
cal strategies have been deployed to alleviate the consequences 
of such decisions. Several authors have thresholded correlation 
matrices by applying an inferential cut-off point. This approach 
is similar in spirit to the standard mass univariate strategy regu- 
larly adopted within the context of classical statistical parametric 
mapping (Friston, 1994). 

However, this thresholding of matrices is generally critized 
for throwing away valuable information. Indeed, since network 
analysis proceeds by comparing the global topological properties 
of the graphs obtained after binarizing correlation matrices, it 
is natural to conclude that a substantial amount of real-valued 
information has been discarded; and replaced by a sequence of 
binary digits. As a result, several authors have proposed to use the 
weighted versions of the classical graph-theoretical measures of 
topology (Rubinov and Sporns, 2010). It is commonly believed 
that the use of such weighted topological statistics alleviates both 
the problem of selecting an arbitrary threshold, and also ensures 
that one is separating differences in topology from differences in 
network density. Although this first requirement is indeed satis- 
fied, the second is only illusory. We will show in this paper that the 
use of weighted topological measures is just as liable to be deter- 
mined by differences in density, as their standard unweighted 
versions. 

In the present paper, we will concentrate our attention on 
weighted networks since these are more likely to be found in 
the biomedical sciences than their unweighted counterparts. 
This article is structured in two parts. We firstly review how 
to construct summary networks representing subject- specific or 
group-specific functional connectivity over time. Here, a mass- 
univariate approach is adopted using different corrections for 
multiple comparisons. A similar approach can also be used for 
representing group differences in functional network topologies. 
In a second part, we concentrate on network properties inference. 
This is rendered particularly arduous by the fact that such net- 
works tend to display different number of edges. Since network 
density is highly predictive of a host of network topological mea- 
sures, such statistical inference requires special attention, when 
comparing groups of subjects that exhibit substantial differences 
in network density. 

2. CONSTRUCTION OF SUMMARY NETWORKS 

We firstly describe how one can construct summary networks 
from a family of subject- specific weighted or unweighted net- 
works. This task can be tackled by combining the data available, 
using a mass -univariate approach, as is commonly done in fMRI. 



Note that the phrases, graph and network, will be used inter- 
changeably in this paper. 

2.1. STATISTICAL PARAMETER NETWORK (SPN) 

Here, we review an efficient method for summarizing inference 
on networks, using a mass-univariate approach. By tacit consen- 
sus, this method has essentially become the norm in the field 
(Achard et al, 2006; He et al., 2007, 2009b; Ginestet et al, 2012). 
This strategy should be compared to the one adopted in the clas- 
sical statistical parametric mapping (SPM) framework, which has 
been utilized in neuroimaging for the past two decades (Friston, 
1994). Consequently, this approach will be referred to as statis- 
tical parametric networks (SPNs). The problem of constructing 
a summary graph centers on how to combine the elements of a 
population of subject- specific correlation matrices. In the SPN 
framework, summary networks are constructed irrespective of 
whether or not structural or functional data are being used. While 
in fMRI studies, it has been common for researchers to compute 
correlations over time between regions of interest (Achard et al, 
2006; Achard and Bullmore, 2007), studies based on structural 
MRI data, by contrast, have considered between- regions correla- 
tions with respect to the available population of subjects (Bassett 
et al., 2008). In this section, we will concentrate on the specific 
problem posed by the study of functional MRI cortical net- 
works, where each subject-specific correlation matrix represent 
inter-regional normalized covariances, computed with respect to 
a sequence of time points. 

Succinctly, one may say that an SPN is to a correlation matrix, 
what an SPM is to an intensity map. As for the latter, an SPN can 
be produced in order to obtain a summary network. Different 
summary networks can be constructed for the different condi- 
tions of an experiment, or for the different groups of subjects 
under scrutiny. Achard et al. (2006) and He et al. (2009b), for 
instance, have visualized their data using summary networks, 
whereby an edge is solely included when a corresponding test 
statistic for that edge is significant. We will refer to such summary 
networks as mean SPNs. Similarly, one can construct differen- 
tial or difference SPNs, which represent the edges that have been 
significantly "lost" and the edges that have been significantly 
"gained," when comparing the graphs across experimental con- 
ditions, or when considering several groups of subjects. Under its 
many guises, this approach has been adopted by various authors 
including Zalesky et al. (2010) and Richiardi et al. (2011), who 
have used network-based statistics and machine learning meth- 
ods, respectively, for the comparison of a group of subjects with a 
group of controls. 

The SPN approach that we wish to present here is slightly 
more general, since it accommodates sophisticated experimental 
designs, in which information may be pooled over a number of 
experimental conditions. As for SPM, such analyses enable a con- 
cise visualization of the data, which can be interpreted in terms 
of network properties, topology and community structure. This 
approach is particularly helpful for an efficient reporting of the 
experimental results. As mentioned in the introduction, the use 
of SPNs has the additional advantage of somewhat alleviating the 
methodological concerns associated with the choice of an arbi- 
trary threshold value; since we are here selecting such cut-off 
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points on the basis of a specific p-value. Network thresholding 
is therefore here supplanted by inference. 

The thresholding of association matrices, such as correlation 
matrices, is equivalent to the application of an elementwise indi- 
cator function. This type of function, however, is non-linear, in 
the sense that the sum of the thresholded correlation matrices is 
not equal to the thresholded mean correlation matrix. That is, this 
may be formally expressed, as follows, 



£t t (r ; ) # r r (£R, 

i = 1 



i=\ 



(1) 



where i = 1, . . . , n labels the subjects taking part in the experi- 
ment, and where R; 's denote subject- specific correlation matrices. 
Here, the function, T T , is a thresholding function that takes 
a matrix, and returns its binarized version, with respect to a 
cut-off point, r. The issue of thresholding correlation matrices 
is illustrated in Figure 1, where we have reported some of the 
preliminary data analysis conducted in Ginestet et al. (2012). 

Currently, there is little guidance on how one should pro- 
ceed, when summarizing the network analysis of a given study. 
There is hence a pressing need to reach a methodological con- 
sensus on how to standardize the construction and reporting of 
summary networks in neuroscience. A natural desideratum for 
such summary networks is that they should reflect the topological 
variability of the entire population of networks. Pioneering work 
in that direction has been laid out by several authors, including 
Achard et al. (2006) and He et al. (2009b), for the consideration 
of a single family of graphs. In the sequel, we review these ideas 
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FIGURE 1 | Illustration of the use of mean SPNs to summarize 
networks in a cognitive task. Adjacency matrices become sparser with 
increasing working memory load. In panel (A), heatmaps corresponding to 
the correlation matrices in each of four A/-back conditions, for n = 43 
subjects. In panel (B), the adjacency matrices were obtained by 
constructing mean SPNs, using a mass-univariate approach based on 
z-tests with respect to the grand sample mean r and the grand sample 
standard deviation sd(r) with FDR correction (base rate ao = .05). Zero 
entries are denoted in black in the adjacency matrices. (See Ginestet and 
Simmons, 2011 for a full description.) 



and extend them to the case of several populations of networks, 
as was conducted in Ginestet et al. (2012). 

The question of drawing inference on families of networks that 
vary over several experimental conditions can be subdivided into 
two related issues. On the one hand, one needs to test whether 
or not the properties of the nodes have been significantly affected 
by the experimental manipulation. On the other hand, one also 
needs to evaluate whether or not the presence and absence of 
edges have significantly varied across the experimental condi- 
tions. One can drawn statistical inference for these two distinct, 
yet related, research questions. Contrary to the classical SPM 
framework, these two distinct problematics need to be answered 
using two different types of networks: one for comparing vertices, 
and another for comparing edges. 

A substantial advantage of the SPN methodology is that it 
addresses the problem arising from the quasi-linearity of the 
thresholding function presented in Equation (1). Indeed, since 
we are drawing inference using the correlation coefficients per se, 
we consequently bypass the problem of averaging over a set of 
thresholded correlation matrices; while nonetheless producing a 
statistical summary taking the form of a graph. 

We here employ standard graph theoretical notation in order 
formulate our approach to this specific problem. The interested 
reader is invited to consult Bollobas (1998) for a more solid intro- 
duction to graph objects and their properties. As aforementioned, 
we will here use the terms networks and graphs interchangeably. 
In the context of discrete mathematics, a graph G is formally 
defined as an ordered pair of sets (V,E); in which V(G) repre- 
sents the set of vertices (sometimes referred to as nodes) in the 
graph of interest; whereas E(G) denotes the set of edges in that 
network (also called connections). The total number of edges and 
total number of nodes in G will be concisely denoted by Ne and 
Ny, respectively. A one-way experimental design may be typi- 
cally composed of / experimental conditions, with n subjects, per 
experiment. Thus, the full data set of interest can be described 
as an (n x /) -matrix of correlation matrices. In the sequel, the 
indexes i = 1, . . . , n will label the experimental subjects; whereas 
the indexes; = 1, . . . , / will refer to the experimental conditions. 
Formally, one could represent the full data set as the following 
matrix, 



R = 



Rn ... Ri 



R M i . . . R n j 



(2) 



Here, each element Ry in this equation denotes a correlation 
matrix of dimension Ny x Ny. There is a one-to-one correspon- 
dence between each of these correlation matrices and a weighted 
graphs on Ny vertices or nodes. The individual vertices will be 
labeled by v = 1, . . . , Ny. Moreover, for convenience, each of the 
matrix entries in R, will be denoted by r?-; where the superscript 
e labels an edge from the saturated or complete graph, which 
possesses the maximal number of possible edges. That is, the sat- 
urated graph has the following edge set size, Ny(Ny — l)/2. In 
the rest of this paper, edges will be systematically referred to by 
using superscripts. 
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A mean or summary SPN allows to statistically infer the "aver- 
age" set of inter-regional connections in a group of subjects. 
Such SPNs are generally obtained by adopting a mass -univariate 
approach, whereby a sequence of statistical tests are performed for 
each edge in the edge set. Such an operation may be repeated for 
each experimental condition. Using the notation introduced ear- 
lier, one may conduct a test for each of the columns in the array, 
denoted R, in Equation (2). In effect, we are here considering the 
following column vectors of correlation matrices, 



Rj — [Rij, . . • , R«j] . 



(3) 



Each of these column vectors is analyzed independently in order 
to produce a single network for each of the different experimen- 
tal conditions. For the case of correlation matrices, the original 
matrix entries are routinely Fisher z-transformed, in order to be 
able to use central limit theorems for approximating the density 
functions of these test statistics. In doing so, one can then draw 
inference, using an analysis of variance, for instance, or another 
adequate statistical model, suitable for the data at hand. An exam- 
ple of such mean SPNs under different experimental conditions is 
reported in Figure 2. 

Perhaps, the tenor research question in network data analy- 
sis in neuroscience is whether certain edges have been "gained" 
or "lost," as a consequence of a particular experimental condi- 
tion. This general research question can be specifically answered 
by computing two distinct differential networks, representing 
what we may call the downweighted and upweighted SPNs. These 



two types of networks will be denoted by SPN_ and SPN+, 
respectively. 

As for mean SPNs, the construction of these differential 
networks can similarly be conducted within a mass -univariate 
approach. For differential SPNs, however, statistical inference 
needs to be drawn from the full data set. That is, one needs 
to consider all the correlation coefficients described in Equation 
(2) -that is, the elements contained in the matrix R. Computing 
a differential SPN will generally involve Ne linear models. 
Depending on the general experimental framework adopted by 
the researchers, these linear models could be extended to mixed 
effects models. In its most general formulation, we may consider a 
repeated block design, which can be succinctly expressed by using 
the classical formalism due to Laird and Ware (1982), 



x\ = X^ e + + ef ; 



l,...,n. 



(4) 



Here, each vector, r • = jV^ , . . . , r^J , denotes the correlation 

coefficients of interest, and f3 e = , . . . , /3j] T consists of the 
vector of fixed effects. The latter does not vary over subjects 
and will be the main object of study. By contrast, the b^'s are 
the vector of subject-specific random effects, which will be inte- 
grated over. Finally, e\ = ^ , . . . , J is the vector of residuals. 
Crucially, the X/s and Z/s denote the design matrices for the fixed 
and random effects, respectively. As in standard applications of 
mixed effects models, the co variance matrices for e e and b e can 
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FIGURE 2 | Graphical representations of mean SPNs over four levels of a 
cognitive task. The mean SPNs for an A/-back task, in the coronal (A) and 
transverse (B) planes are here presented, after FDR correction (base rate 
ao = 0.05). Locations of the nodes correspond to the stereotaxic centroids of 



the corresponding cortical regions. The orientation axes are indicated in 
italics: inferior-superior and anterior-posterior for the coronal and transverse 
sections, respectively. The size of each node is proportional to its degree. 
(See Ginestet and Simmons, 2011 for a full description.) 
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be assumed to be diagonal and positive semi-definite, respectively 
(see Demidenko, 2004, for details). 

In general, one may include an edge in a differential SPN, 
when the corresponding F-test for the experimental factor has 
been found to be significant. Depending on the linear model 
used, different statistical test may be performed (Pinheiro and 
Bates, 2000). Therefore, the use of a mass -univariate approach 
for extracting between-condition differences in the presence or 
absence of edges, yields two different types of differential SPNs. 
That is, depending on the sign of the significant fixed effect coef- 
ficients, one may include that edge in either a downweighted 
network, which may be denoted SPN_; or in an upweighted 
network, denoted SPN+. 

A similar approach can be adopted to estimate the upweight- 
ing and downweighting of the signal of interest at single nodes. 
Again, such a node-specific differential SPN can be obtained by 
performing a set of Ny linear models. In this case, the data under 
consideration is the set of matrices Y v = {yV.}, where each v e V 
is a region of interest. Every y~ corresponds to a time-averaged 
intensity signal, for the vth region, for subject i, under the jth 
experimental condition. Thus, one could reformulate the system 
of equations for evaluating edges in (4) by using superscripts to 
denote vertices. 

As for edge-specific differential SPNs, a vertex would be esti- 
mated to be either significantly upweighted or downweighted, 
depending on the sign of the largest coefficient in the correspond- 
ing vector f3 v . An illustration of such a differential SPN, based on 
the iV-back data set, analyzed by Ginestet et al. (2012) is reported 
in Figure 3. Naturally, this assignment based on the sign of the 
fixed effects is only possible, when the task under scrutiny is 
based on an experimental gradient. An alternative strategy may 
be required, when different levels of the task are expected to affect 
the response in different directions. 

A singular limitation, however, affects all mass -univariate 
approaches. Such a repetitive use of classical inferential thresh- 
old, may lead to a corresponding increase in Type I error. This 
issue can be addressed by correcting for multiple comparisons. 
The significance of edges and nodes in both mean and differential 
SPNs can, for instance, be inferred using the false discovery rate 



(FDR) with a base rate of ao = .05 (Benjamini and Hochberg, 
1995; Nichols and Hayasaka, 2003). Naturally, other corrections 
for multiple comparisons could also be utilized (see Meskaldji 
et al., 2011, for a different approach). The conventional thresh- 
olding method used in network analysis is therefore superseded 
by the application of standard multiple testing corrections. The 
main advantage of this approach lies in its pooling of informa- 
tion over several subjects, in order to produce robust edge- and 
node-specific statistics. 

3. COMPARISON OF FUNCTIONS ON NETWORKS 

We now turn to the issue of comparing various types of topolog- 
ical measures over several families of networks (van Wijk et al., 
2010). Inference on quantities such as characteristic path length, 
clustering coefficient and modularity structure has attracted a 
sustained amount of interest in the neuroscience community. 
Comparisons of this type of topological measures, however, is 
generally regarded to be hard, since these topological differences 
highly depend, in a non-linear fashion, on group differences in 
edge density. 

3.1. GLOBAL EFFICIENCY 

One of the classical exemplars of a topological summary of a net- 
work is its characteristic path length. Such a quantity, however, 
is solely defined for connected graphs. The global efficiency of a 
graph, by contrast, can be computed for any network -connected 
or disconnected- and is inversely related to its characteristic path 
length. Efficiency is formally defined by the following formula due 
to Latora and Marchiori (2001), 

E (G) = r T T dT.\ (5) 

N V (N V - 1 ^ ^ v 

with Ny = \ V\, as before. Here, djj denotes the length of the 
shortest path between vertices i and;. Moreover, the second sum- 
mation is performed with respect to the set, {j ^ i e V}, which is 
the set of all indices in V that are different from i. This efficiency 
measure can be shown to be equivalent to the inverse of the har- 
monic mean of the length of the shortest paths between each pair 
of nodes in the network G. 

Specifically, the quantity in Equation (5), is usually referred 
to as the global efficiency of a particular graph, and is denoted 
by £Glo(G) = E(G). Intuitively, this quantity can be understood 
as the amount of potential information transfer that can be per- 
formed in parallel. A local measure of efficiency can also be 
computed, which is equivalent to the clustering coefficient. For 
a review of other efficiency measures that have been studied in 
the context of neuroscience, the reader is referred to Ginestet 
et al. (2011). The most commonly adopted approach to net- 
work comparison is therefore to compute a topological metric, 
such as global efficiency, for each individual subject, and there- 
after to evaluate whether this measure differs over the different 
experimental groups under scrutiny. 

3.2. DENSITY-INTEGRATED MEASURES 

An alternative approach to the problem of quantifying the topol- 
ogy of weighted networks proceeds by integrating the metric 




L R 

FIGURE 3 | Visualization of a differential SPN, summarizing the effect 
of a cognitive experimental factor. Sagittal section of a negative 
differential SPN, which represents the significantly "lost" edges, due to the 
A/-back experimental factor. The presence of an edge is determined by the 
thresholding of p-values at 0.01, uncorrected (see Ginestet and Simmons, 
2011, for a description of the data at hand.). 
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of interest with respect to different density levels. Different 
approaches have been adopted in practice. While some authors 
have integrated over a subset of the density range (see Achard 
and Bullmore, 2007, for example), others have integrated over the 
entire range of densities (He et al., 2009a). The family of topolog- 
ical measures, which is obtained after integrating over different 
density levels, will be referred to as density-integrated measures. 
Given a weighted graph G = (V, £ , W), the density-integrated 
version of the efficiency in Equation (5) can, for instance, be 
defined as follows, 

E K (G) = j E(y(G,k))p(k)dk, (6) 

where density is treated as a discrete random variable K, with real- 
izations in lower case, and p(k) denotes the probability density 
function of K. Since K is discrete, it can only take a countably 
finite number of values. In general, it is common to assign equal 
weight to every possible choice of density. 

The function y(G, k) in Equation (6) is a density-thresholding 
function, which takes a weighted undirected network and a level 
of wiring density as arguments, and returns an unweighted net- 
work. Since there is no prior knowledge about which values of K 
should be favored, one can specify a uniform distribution on the 
set of all possible densities. Note, however, that other distributions 
could be selected for this purpose (see Ginestet et al., 2011, for a 
discussion of alternative specifications). 

3.3. INTEGRATING OVER DENSITIES 

The question of separating topology from density could be 
reformulated as the statistical problem of evaluating topo- 
logical differences, while "controlling" for differences in den- 
sity. When adopting this perspective, it is convenient to treat 
topology and density as random variables. We have already 
done so for density, in the previous section. Implicitly, by 
integrating over all possible thresholds, we are indeed con- 
sidering density as a random variable with a well-defined 
probability distribution, which is, in the present case, a uniform 
distribution. 

A natural desideratum, which may be required when com- 
paring network topological characteristics, while controlling for 
differences in topology; would be to control for weighted net- 
works whose association matrices are proportional to each other. 
That is, if two different matrices are linearly related to each other, 
it seems reasonable to conclude that their topologies must be 
identical, after one has controlled for such a linear difference in 
density. Thus, consider the following simple example, adapted 
from Ginestet et al. (201 1). 

Example 1 (Ginestet et al., 2011). We here have two networks, 
G\ and G2, with proportional association matrices Wi and W2, 
satisfying Wi = aW 2 . That is, these two matrices are propor- 
tional to each other. An application of the density-integrated 
metrics described in Equation (6) to these networks would give 
the following equalities, 

E*(Wi) = E K (aW 2 ) = E K (W 2 ) . (7) 



That is, when integrating with respect to density, we are in fact 
evaluating the efficiencies of G\ and G2 at a number of cut-off 
points. At each of these points, the efficiency of the two networks 
will be identical, because Wi is proportional to W2 and therefore 
the same sets of edges will be selected. Therefore, G\ and G2 have 
identical density-integrated efficiencies. 

While illustrative, this example is not entirely satisfying. In 
fact, this result can be shown to hold in a more general sense. The 
invariance of density-integrated efficiency turns out to be true for 
any monotonic (increasing or decreasing) function h, as formally 
stated in the following result. 

Proposition 1 (Ginestet et al, 2011). Let a weighted undirected 
graph G = (V, £ , W). For any monotonic function h( • ) acting ele- 
mentwise on a real-valued matrix W, and any topological metric E, 
the density -integrated version of that metric, denoted Ek, satisfies 

E K (W) = E K (h(W)), (8) 
where we have used the weight set, W, as a proxy for graph G. 

A proof of this proposition can be found in Ginestet et al. 
(2011). The demonstration essentially relies on the fact that any 
monotonic transformation of the entries of a real-valued matrix 
will preserve the ranks. Therefore, proposition 1 makes rigorous 
a potential way of "controlling" for differences in density. That 
is, this formal proposition states that we are indeed controlling 
for any monotonic transformation of the original entries in the 
matrix. In effect, proposition 1 should be regarded as a potential 
definition of what it means for two networks to solely differ in 
terms of topology, while controlling for monotonic differences in 
density. 

3.4. DENSITY AND MODULARITY 

Another network property, which has been studied extensively in 
the literature is modularity structure. As for efficiency and other 
topological measures, however, modularity is also highly depen- 
dent on edge density. Therefore, any attempt at comparing the 
modularity of different groups of networks will be confounded 
by group differences in the networks' number of edges. We illus- 
trate this problem with the results reported in a recent paper by 
Bassett et al. (2011), who have analyzed the static and dynamic 
organization of functional brain networks in humans. We here 
focus on the first claim made in this paper, which states that the 
static modular structure of such networks is nested with respect 
to time. In particular, Bassett et al. (2011) argue that this graded 
structure underlines a "multiscale modular structure." 

As for global efficiency in the previous section, it can be shown 
that modularity structure is substantially mediated by edge den- 
sity. In the case of weighted networks, this is equivalent to a 
difference in the size of the correlation coefficients. In Bassett et al. 
(2011), for instance, the authors report that the size of the mean 
correlation diminishes with the size of the time window. Such 
a decrease in overall correlation will generally have two effects: 
(i) networks' topologies will become increasingly more "random" 
and (ii) the number of significant edges will decrease. Here, we 
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use synthetic data sets to show that these two phenomena are 
likely to be associated with a higher number of modules, thereby 
potentially explaining the apparent multiscale modular structure 
described by Bassett et al. (2011). Our simulations are based on 
the unweighted unsigned version of the modularity algorithm of 
Clauset et al. (2004), but may be extrapolated to weighted signed 
adjacency matrices. 

In Figure 4A, we have generated 1000 unweighted lattices 
based on 112 vertices as in Bassett et al. (2011). By randomly 
rewiring the edges of these lattices, we show that the number 
of modules in these networks tends to increase with the level 
of topological randomness in these graphs. For Figures 4B-D, 
we have generated two sets of unweighted networks, charac- 
terized by a random and a regular topology, respectively, with 
different number of edges. These simulations were repeated 
1000 times for each type of graph for each number of edges. 
For both types of networks, the number of modules in these 
graphs tended to decrease as new edges were added. Collectively, 



although these data simulations do not entirely rule out the 
possibility of a temporally nested modular structure in the 
human brain, they nonetheless cast doubts on the possibil- 
ity of detecting such a temporal organization by reducing 
the size of the sampling window. Such subtle artifactual rela- 
tionships between modularity and edge density can arise in 
a range of different settings in the analysis of neuroimaging 
data. 

3.5. WEIGHTED TOPOLOGICAL METRICS 

Since the previous two sections have highlighted the potential 
problems associated with thresholding correlation matrices, one 
may surmise that such problems could be adequately dealt with, 
by directly considering the weighted versions of the topological 
metrics of interest. In particular, an apparently natural way of 
combining differences in density with differences in topology is 
to consider the weighted versions of traditional topological met- 
rics. For the aforementioned global efficiency, for instance, one 
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FIGURE 4 | Topological randomness and number of edges predict 
number of modules. (A) Relationship between the number of random 
rewirings of a regular lattice and the number of modules in such a network. 
Here, the number of edges is kept constant throughout all rewirings. (B) 
Relationship between the number of edges in a network and its number of 
modules for both regular (i.e., lattice) and random graphs. This shows that the 



number of modules tends to decrease as more edges are added to both 
types of networks. (C,D) Modular structures of regular (C) and random (D) 
networks for different number of edges, Ne- These networks are 
represented using the algorithm of Kamada and Kawai (1989) with different 
colors representing different modules. In all simulations, the number of 
vertices is N v = 112, as in Bassett et al. (2011). 
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can define a weighted global efficiency, denoted as follows, 



1 Nv Nv 1 1 1 

Ew(G) = N v (N v -\) E E = ^ E 



(9) 



where represents the weighted shortest path between the ith 
and jth nodes. Unfortunately, another theoretical result points 
to a serious limitation of Ew, which may potentially dissuade 
researchers from using this particular type of metrics. With the 
next proposition, we demonstrate that under mild conditions, the 
weighted efficiency is simply equivalent to the weighted density, 
sometimes referred to as weighted cost, of the graph of interest, 



K W (G) 



1 



N v (N v - 1) 



N v N v 

EE^- 



(10) 



Proposition 2 (Ginestet et al., 201 1). For any weighted graph G = 
(V, £ , W), whose weighted edge set is denoted by W(G) = [wij 
i<j),if 



mm w^ > 

w„-eW(G) J 



1 

- max 1 

2 WyeVv(G) 



then 



Ew(G) = Kw(G). 



(11) 



(12) 



A proof of this result can be found in Ginestet et al. (2011). 
Not surprisingly, proposition 2 places emphasis on the spread of 
the distribution of the weighted edge set £ (G). The condition in 
proposition 2 may at first appear quite constraining. However, 
this condition encompasses a wide range of experimental situa- 
tions, including the data set described in Ginestet and Simmons 
(2011). Thus, the added benefit of utilizing the weighted ver- 
sion of the global efficiency measure may, in most settings, be 
highly questionable, since there exists a one-to-one relationship 
between this topological measure and a simple average of the edge 
weights. Cutoff-integrated efficiency and other cutoff-integrated 
measures, as described in Ginestet et al. (2011), may therefore 
be preferred, in practice, when one wishes to summarize the 
influence of both density and topological differences. 

4. CONCLUSION 

In this paper, we have briefly reviewed some of the methodolog- 
ical research that has been conducted on network data analy- 
sis, as applied to functional neuroimaging. Two main threads 
ran through this discussion. Firstly, we considered the differ- 
ent approaches that one may adopt, when summarizing several 
subject- specific networks. Secondly, the thorny issue of graph 
thresholding was also tackled, with special emphasis on the 
comparison of network modularity and the use of weighted 
topological metrics. 

From the above discussion, it should be clear that there does 
not exist a single way of computing a mean network. This is, 
in some sense, an ill-defined problem. A commonly adopted 



perspective on this issue is to perform a mass -univariate test, 
where the significance levels of every edge are evaluated, and then 
thresholded. We have seen that this approach can be carried out 
both within a single family of networks, and over an entire exper- 
imental design, using a mixed effects model. By analogy with the 
classical SPM approach used in neuroimaging, one may refer to 
such uses of a mass -univariate approach on networks, as SPNs. 

Secondly, we have discussed one of the long-standing issues 
in the application of network data analysis to neuroscience data: 
the question of whether or not one should threshold matrices of 
correlation coefficients, for the purpose of producing adjacency 
matrices. In this paper, we have reviewed a range of different 
approaches to this problem. On the basis of the several exam- 
ples and counterexamples that we have studied, we are able to 
make a few methodological recommendations to researchers in 
the neuroscience community, intending to compare the topolog- 
ical properties of two or more populations of weighted networks. 
Note that these recommendations are solely tentative, as no 
general consensus has yet been reached on this particular issue. 

As a first step, we argue that it is good practice to standardize 
the association weights. This may facilitate comparison across dis- 
tinct network analyses, and ease the interpretation of the results. 
Secondly, the weighted density, or connectivity strength, of the 
networks of interest should then be reported. This is central to 
the rest of the analysis, and therefore, this quantity should be 
computed and reported systematically. Indeed, if the groups of 
networks under scrutiny substantially differ in terms of average 
density, then these differences are highly likely to affect any com- 
parison of the topological properties of these groups of networks. 
Finally, population differences in density-integrated topological 
metrics may then be evaluated and reported. This will indicate 
whether the topologies of the populations under scrutiny vary 
significantly after having controlled for monotonic differences in 
connectivity strength. 

The theoretical results described in this paper have only been 
presented for the global efficiency metric. Thus, these proposi- 
tions and the examples studied need not necessarily apply to other 
topological measures. However, we also note that proposition 1 
has been proved with a high degree of generality. This proposi- 
tion and its proof is indeed independent of the particular formula 
of the metric of interest, and therefore could easily be extended 
to any other function of the weighted graph matrix. In particu- 
lar, because most weighted metrics are constructed on the basis 
of the matrix of weighted shortest paths, one surmises that this 
theoretical result may, in fact, hold in a more general setting. 

Importantly, we have also shown that network modularity is 
not immune to this dependency on edge density. If several pop- 
ulations of networks differ in their number of edges, then it is 
likely that the resulting group -specific modularity structures will 
not be comparable. That is, such comparisons will mainly reflect 
differences in edge density, and as such may not carry much 
explanatory power. This is an area of application of statical net- 
work analysis, where one should exert caution, as the powerful 
algorithms used for detecting network modules may hide the 
potential confounding effects of differences in edge density. 

Finally, the use of weighted topological metrics was also 
considered. Unfortunately, we have seen that simply replacing 
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classical network measures by their weighted analogs is not suffi- 
cient to resolve the dependency of these measures on edge density. 
Thus, cutoff- integrated topological measures, such as the cutoff- 
integrated efficiency described in Ginestet et al. (2011), may be 
preferred in practice, when one wishes to separate differences in 
edge density from differences in topology. 
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